Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate use of initial increment/perturbation file from REPLAY/+03 ICs #3119

Merged
merged 114 commits into from
Jan 8, 2025
Merged

Separate use of initial increment/perturbation file from REPLAY/+03 ICs #3119

merged 114 commits into from
Jan 8, 2025
+36 −35

Conversation

NeilBarton-NOAA
Copy link
Contributor

Description

This PR is a continuation of the previous SFS PR. The use of initial perturbations/increments for ensemble long forecasts (e.g., SFS and GEFS) was missing from the previous PR because this option was included with the REPLAY_IC option.

In this PR, the REPLAY_IC option is now strictly for the +03 ICs, and the variables USE_ATM_ENS_PERTURB_FILES and USE_OCN_ENS_PERTURB_FILES are used for initialization with the initial perturbation/increment files used for SFS and GEFS reforecasting. This separation will also make it easier to remove the +03 IC optoin.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

I tested long forecast runs on hercules and multiple CI tests.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

NeilBarton-NOAA and others added 30 commits July 2, 2024 15:04
@NeilBarton-NOAA
initial creation of CI using replay ICs
30cf5df
@NeilBarton-NOAA
change name
4b3ae39
@NeilBarton-NOAA
making an S2SWA case
fe2054e
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
7b7ee9b
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
225d280
@NeilBarton-NOAA
start of CI for GEFS replays # Please enter the commit message for your
b17765e 
changes. Lines starting
@NeilBarton-NOAA
Merge branch 'develop' into gefs_replay_ci
675cb77
@NeilBarton-NOAA
removing shell check errors
e49dbcf
@NeilBarton-NOAA
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
ec4becc 
…lobal-workflow into gefs_replay_ci
@NeilBarton-NOAA
fixes for wave post processing
b69d653
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
084d744
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
a4bb8f3
@NeilBarton-NOAA
add BREAKPOINTS variable
4ff31fd
@NeilBarton-NOAA
fix shell checks
97b92ee
@NeilBarton-NOAA
edit defult config.efc to be close to re-forecasting
713f41d
@EricSinsky-NOAA
Fix bug in variable check
af31d2f 
Fix a bug in a variable check in the extractvars task where parenthesis in the variable name will prompt a false warning message.
@EricSinsky-NOAA
Add more settings to GEFS replay CI test
cec7b81
@EricSinsky-NOAA
Add atmos variables requested for reforeacst
4aaca9a
@EricSinsky-NOAA
Reorganize TSNOWP in paramlist
11e1957
@EricSinsky-NOAA
Extract ice variables from history file instead
a005e5f 
Ice variables are extracted from the history files instead of the product files. This is necessary because there are certain ice variables that are needed that are only available in the history files and not the product files.
@EricSinsky-NOAA
Fix typo
2dd9a7e
@EricSinsky-NOAA
Update TCDC variable to correct name
b132408
@NeilBarton-NOAA
Merge pull request #7 from EricSinsky-NOAA/gefs_replay_ci_pr
3d9455b 
Add suggested patches to gefs_replay_ci
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
447224c
@NeilBarton-NOAA
edits to stage_ic for replay perturbations
c6b7eaa
@NeilBarton-NOAA
adding icsdir to pr CI yaml
51aa2dd
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
a641591
@EricSinsky-NOAA
Fix ensstat failures
4985361
@NeilBarton-NOAA
updates for products
8eaef27
@EricSinsky-NOAA
Add shellcheck disable and skip f000 for ocn/ice
2e7241f 
Shellcheck disable has been added. Also, f000 is skipped in extractvars for ocn and ice since there are no products for f000 for ocn and ice. This change will remove warning messages.
@aerorahul aerorahul added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Jan 7, 2025
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jan 7, 2025
@emcbot
Copy link

emcbot commented Jan 8, 2025

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/3119


Experiment C48mx500_3DVarAOWCDA_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 23:09:53 UTC 2025
Experiment C48_ATM_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 23:16:11 UTC 2025
Experiment C48mx500_hybAOWCDA_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 23:16:31 UTC 2025
Experiment C96_S2SWA_gefs_replay_ics_ab90c37b Completed 1 Cycles: *SUCCESS* at Tue Jan  7 23:28:47 UTC 2025
Experiment C96C48_hybatmDA_ab90c37b Completed 3 Cycles: *SUCCESS* at Wed Jan  8 00:24:15 UTC 2025
Experiment C96_atm3DVar_ab90c37b Completed 3 Cycles: *SUCCESS* at Wed Jan  8 00:30:25 UTC 2025
Experiment C96C48_hybatmaerosnowDA_ab90c37b Completed 3 Cycles: *SUCCESS* at Wed Jan  8 00:30:25 UTC 2025
Experiment C96C48_ufs_hybatmDA_ab90c37b Completed 3 Cycles: *SUCCESS* at Wed Jan  8 01:08:02 UTC 2025
Experiment C48_S2SW_ab90c37b Completed 2 Cycles: *SUCCESS* at Wed Jan  8 01:33:12 UTC 2025
Experiment C48_S2SWA_gefs_ab90c37b Completed 1 Cycles: *SUCCESS* at Wed Jan  8 02:05:11 UTC 2025

@emcbot emcbot added CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Jan 8, 2025
@emcbot
Copy link

emcbot commented Jan 8, 2025

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3119


Experiment C48_ATM_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 16:40:42 CST 2025
Experiment C48mx500_3DVarAOWCDA_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 16:52:45 CST 2025
Experiment C48mx500_hybAOWCDA_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 16:59:05 CST 2025
Experiment C96_S2SWA_gefs_replay_ics_ab90c37b Completed 1 Cycles: *SUCCESS* at Tue Jan  7 17:30:07 CST 2025
Experiment C96_atm3DVar_ab90c37b Completed 3 Cycles: *SUCCESS* at Tue Jan  7 17:47:37 CST 2025
Experiment C96C48_hybatmDA_ab90c37b Completed 3 Cycles: *SUCCESS* at Tue Jan  7 17:47:51 CST 2025
Experiment C48_S2SW_ab90c37b Completed 2 Cycles: *SUCCESS* at Tue Jan  7 18:06:15 CST 2025
Experiment C48_S2SWA_gefs_ab90c37b Completed 1 Cycles: *SUCCESS* at Tue Jan  7 20:21:45 CST 2025

@emcbot emcbot added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Jan 8, 2025
@emcbot
Copy link

emcbot commented Jan 8, 2025

CI Tests set up to run in /lfs/h2/emc/ptmp/emc.global/PR/PR_3119/RUNTESTS on WCOSS

@emcbot emcbot added the CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS label Jan 8, 2025
@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress label Jan 8, 2025
@emcbot emcbot added CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Jan 8, 2025
@emcbot
Copy link

emcbot commented Jan 8, 2025

CI Tests set up to run in /lfs/h2/emc/ptmp/emc.global/PR/PR_3119/RUNTESTS on WCOSS

@KateFriedman-NOAA
Copy link
Member

CI tests on WCOSS2 are complete and successful:

Wed Jan  8 17:34:37 UTC 2025
******** C48_ATM_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 08 2025 08:20:19    Jan 08 2025 09:30:27
202103231800        Done    Jan 08 2025 08:20:19    Jan 08 2025 09:40:20

******** C48_S2SW_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 08 2025 08:20:22    Jan 08 2025 10:05:23
202103231800        Done    Jan 08 2025 08:20:22    Jan 08 2025 10:15:31

******** C48_S2SWA_gefs_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 08 2025 08:20:27    Jan 08 2025 11:00:33

******** C96_atm3DVar_extended_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Jan 08 2025 08:20:32    Jan 08 2025 08:40:42
202112210000        Done    Jan 08 2025 08:20:32    Jan 08 2025 12:45:39
202112210600        Done    Jan 08 2025 08:20:32    Jan 08 2025 13:35:40
202112211200        Done    Jan 08 2025 08:45:52    Jan 08 2025 14:25:39
202112211800        Done    Jan 08 2025 12:50:35    Jan 08 2025 17:30:33

******** C96C48_hybatmDA_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Jan 08 2025 08:20:35    Jan 08 2025 08:40:47
202112210000        Done    Jan 08 2025 08:20:35    Jan 08 2025 10:50:50
202112210600        Done    Jan 08 2025 08:20:35    Jan 08 2025 10:35:53

******** C96_S2SWA_gefs_replay_ics_3119 ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202011010000        Done    Jan 08 2025 08:20:38    Jan 08 2025 09:00:51

Setting label to passed for WCOSS2.

@KateFriedman-NOAA KateFriedman-NOAA added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Jan 8, 2025
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit bb029b5 into NOAA-EMC:develop Jan 8, 2025
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@XiaqiongZhou-NOAA XiaqiongZhou-NOAA XiaqiongZhou-NOAA left review comments

@KateFriedman-NOAA KateFriedman-NOAA KateFriedman-NOAA left review comments

@RuiyuSun RuiyuSun RuiyuSun approved these changes

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@EricSinsky-NOAA EricSinsky-NOAA EricSinsky-NOAA left review comments

@JessicaMeixner-NOAA JessicaMeixner-NOAA JessicaMeixner-NOAA left review comments

@aerorahul aerorahul Awaiting requested review from aerorahul aerorahul is a code owner

@junwang-noaa junwang-noaa Awaiting requested review from junwang-noaa

@DavidHuber-NOAA DavidHuber-NOAA Awaiting requested review from DavidHuber-NOAA

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA Awaiting requested review from TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA is a code owner

Assignees
No one assigned
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.